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(57) A face (skin color) area of a person is extracted 
at a minimized error irrespective of different skin colors 
of the human races. A hue value calculating portion (1) 
determines a hue value from an input image (RGB sig- 
nal). A primary discriminating portion (2) extracts pixels 
whose hue value lies in a specified range defined by lim- 
iting values outputted from a control portion (5). A pixel- 
counting portion (3) counts the extracted pixels. The 
control portion (5) selects a threshold value for extract- 
ing a face area according to the count of the extracted 
pixels and outputs the threshold value to a face-area 
extracting portion (4) which in turn extracts a face area 
according to the threshold value. 

The method and device are usable for video 
processing device, e.g. for producing video information 
suitable to use in video telephones and video confer- 
ences. 
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Description 

BACKGROUND OF THE INVENTION 

The present invention relates to a specified image-area extracting method and a specified image-area extracting 
device, and more particularly, to a method of and a device for extracting, from an input image, a remarkable portion, 
e.g., a person's skin-color portion including its face, arms and the like as a specified image-area, which method and 
device are usable for video processing device, e.g., for producing video information suitable to use in video telephones 
and video conferences. 

Up to now, face-area extraction by detecting a location of a face of a human figure in an input image and processing 
the detected area with priority has been proposed as an effective method to improve the quality of video image. It is 
easily understood that a face portion is remarkable in an image displayed on a display screen of : e.g., a video telephone 
or a video conference apparatus in the field of video communication. It is, therefore, preferred to extract a face area 
from an image and suitably encode and quantize the extracted face area to improve the image quality 

Japanese Laid-Open Patent Publication No. 5-1 651 20, which is referred to as a first example of conventional face- 
area extracting method, discloses sequential steps of a face-area extracting process according to a featured image- 
data extracting method. At 1 st Step, input data R, G, B is cleared from noise components. At 2nd Step, data R, G and 
B areconverted to H (hue) value, S (saturation) value and L (luminance) value. At 3rd Step, a two-dimensional histogram 
showing hue value and saturation value is prepared by using a coordinate system with orthogonal axes for hue value 
(H), saturation value (S) and the number of pixels. At 4th Step, the determined Iwo-dimensional histogram is clustered 
by cutting-out small peaks therefrom by a plane parallel to the coordinate plane and detecting small peaks. At 5th Step, 
a large number of pixels are clustered on the basis of the detected small peaks cutout from the two<Jimensional his- 
togram, and surrounding pixels are integrated together to form an integrated area. The input image scene (frame) is 
divided into areas according to the integrated area. Prospective areas of a person's face are extracted from the divided 
image. At 6th Step, face areas are estimated from the extracted prospective face areas : and then data sets R, G and 
B for estimated face areas are outputted. 

Similar techniques for determining the above-mentioned two-dimensional histogram of parameters H and S and 
dividing and extracting peaks corresponding to respective areas of an image are also proposed in Japanese Laid- 
Open Patent Publication Nos. 6-160993, 6-67320, 5-158164, 5-100328, 4-346334, 4-346333 and 4-346332. Another 
(second) example of a conventional area -extracting method is as follows: 

A two-dimensional histogram is plotted for color difference values U and V, where a face area included in respective 
area. Therefore, the respective area is extracted and outputted as a face area. The above-mentioned processing is 
conducted on all video frames. 

The first example of a conventional face-area extraction method divides an input image into areas by first preparing 
a two-dimensional histogram of hue and luminance and then extracting a peak of histogram frequencies (the number 
of pixels). This method, however, encounters such a problem that it is rather difficult to decide what peak corresponds 
to a skin color area: in practice, the white race and the black race have different hue values, i.e., erroneous face-area 
extraction may arise depending upon the different races. 

The second example of a conventional face-area extracting method involves the following problem: 
Face areas of all races can not be limited to two kinds of color difference distributions and are of at least three 
kinds of color difference (the white race, the black race and the yellow race) because the color difference has a close 
correlation with luminance (brightness). Accordingly, this method can not extract a face area depending upon the race. 
When the method is applied to video processing, a color space of all pixels in each frame must be transformed by 
performing a large number of time-consuming operations. 

SUMMARY OF THE INVENTION 



In view of the foregoing problem of the conventional arts, the present invention was made to provide a skin-color- 
area extracting method that is capable of correctly extracting a skin-color area with no error and with a minimized 
50 amount of processing operations. 

To solve the above-mentioned problems, the present invention provides the following methods and devices: 

(1) A specified image-area extracting method is to derive a value of a color component having a low correlation 
with luminance from each of pixels composing an input colored image, determine distributions of all pixels of the 
input image against each color component value, decide a specified area of each color component value on the 
basis of the determined distributions and extract pixels whose color component value lies within the specified 
range This method can accurately extract a face area of a person in an image with a minimized influence of 
illuminating conditions, skin color difference of the races and variation of skin colors. 
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(2) A specified area extracting method, which relates to the invention of the above (1 ), is further featured by using 
hue values as the color component. This feature is effective to use in practice of extracting a specified-image-area. 

(3) A specified area extracting method, which relates to the invention of the above (1 ) or (2), determines distributions 
of pixels relative to the color component values by preparing a histogram of the color component values, examines 

s whether a peak frequency exceeding a preset value in a preset range of the color component values and selects 

two presetable specified areas on the basis of the examination result. This feature is effective to use in practice 
for extracting a specified image-area from an input image. 

(4) A specified area extracting method, which relates to the above (3), is characterized in that the presetable 
specified range of the color component values is limited by a fixed upper limit value and a fixed lower limit value 

10 to simplify the processing. 

(5) A specified area extracting device comprises arithmetic operation means for calculating a color component 
value having a low correlation with luminance for each pixel in an input colored image, a frame memory for storing 
the input colored image and the color component value determined by the arithmetic processing means, discrim- 
inating means for discriminating whether the determined color component value is included within a specified 

'5 preset range of color components values, counting means for counting-up pixels having color component value 

within the specified range according to the discrimination result, specified image-area extracting means for ex- 
tracting an image within the selected range of color component values read from the frame memory and control 
means for performing control of the above-mentioned means and calculating operations, said control means being 
provided with facilities for comparing a value counted-up by the counting means to a preset value and for selecting 

20 two specified areas of the color component by the specified image-area extracting means. This device is effective 

to use in practice for extracting a specified image-area from an input image. 

(6) A specified area extracting device, which relates to the above (5), is characterized in that an upper limit value 
and a lower limit value limiting the predetermined range of color component values are constant values to simplify 
the processing. 

25 (7) A specified area extracting device comprises a frame memory for storing an input colored image, pixel extracting 

means for deriving a color component value having a low correlation with luminance for each pixel of the input 
image stored in the frame memory, determining distributions of all pixels of the input image according to the color 
component values, determining a specified area of the color component values on the basis of the distributions 
and extracting pixels belonging to the determined specified area, means for preparing a histogram of the extracted 
30 image pixels in an input color-space, means for determining a specified area in the input color-space from the 

distribution of the determined histogram, means for applying the determined specified image area to an input image 
of a proceeding frame and extracting pixels belonging to the determined specified area from the input image, and 
control means for controlling the above-mentioned means and calculations. This device can extract pixels belong- 
ing to a specified area according to a histogram in a color space of an input original image at an increased process- 
es ing rate and therefore is suitable to process images in which a person always exists and will not be altered by 
another. 

(8) A specified area extracting device, which relates to the above (7), is characterized in that the control means 
periodically conduct determination of the specified area in a color space, repeatedly applies the specified area 
determined in the color space to images to be inputted until a new specified area is defined, and extracts pixels 

40 belonging to the specified area. This device can therefore work with a sequence of frames in which a person is 

altered by another. 

(9) A specified area extracting device, which relates to any one of the above (7) and (8), is characterized in that it 
has a facility adapted to process image sequence to be used as the input colored image. This device can be 
effectively used for processing a sequence of video frames. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



Figure 1 is a flow chart for explaining a conventional face-area extracting method. 

Figure 2 is a graph showing a two-dimensional histogram of hue and luminance of an input image according to a 
so conventional face-area extracting method. 

Figure 3 is a graph showing how to divide an input image into areas according to a conventional face-area extracting 
method. 

Figure 4 is a view for explaining a conventional image processing. 
Figure 5 is a graph of histogram of hue values of the white race and the yellow race. 
55 Figure 6 is a graph of histogram of hue values of the black race. 

Figure 7 is a block diagram showing an embodiment of the present invention, which embodiment relates to means 
for realizing a specified image area extracting method. 

Figure 8 is a flow chart showing sequential operations of a control unit of an embodiment shown in Fig. 7. 
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Figure 9 as a block diagram showing a hardware realizing a method of the present invention 

3 bbck f sho ™ 9 an image processing device embod^ng the iresenMnvention 

Figure 1 1 is a view for explaining an embodiment of Fig. 10. invention. 
Figure 12 is a view for explaining an embodiment of Fig. 10. 

PREFERRED EMBODIMENT OF THE INVENTION 



R1 a ndm/«l r T p6rSOnS ,a ° e are extrac «ed from the divided image. At Step S5 face areas 

sisxrji L'issr ,rom ,he extracted prospec,ive ,ace areas ' and — — ~ 

divid^n^e^Sn 6 : TJT^ZVnltr™^ hi ^ram o, parameters H and S and 

Open latent P^^^^^^ ~ ZES£?££7 T 
(second) examp le of a conventional area-extracting method is as follows 4 " 346332 ' An ° ,het 

F.g. 4 shows a two-dimensional histogram plotted for color difference values U and V where a face are* i„H, ,h*h 

35 in de^i! as Slows- 0 *" aCC ° mpanyin9 drawin 9 S ' P referred embodiments of the present invention will be described 
(Embodiment of the present invention, which corresponds to aforementioned (1), (2) (5) or (6)) 
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A hue value calculating portion 1 determines a hue value from an input video signal according to Equation (1). 
Derived hue data together with the input video signal is recorded into a frame memory 6 and then is outputted to a 
primary discriminating portion 2. 

The primary discriminating portion 2 determines a specified range W1 defined by an upper limit value h2 and a 
5 lower limit value hi outputted from a control portion 5 and extracts pixels whose hue values determined by the hue 
value calculating portion 1 lie within the range W1. In this case the upper limit h2 and the lower limit hi may take 
constant (fixed) values. Usually, hi is set at 0.15 rad and h2 at 0.45 rad. The extraction result is outputted to a pixel 
counting portion 3. 

The pixel counting portion 3 counts the number of pixels extracted by the primary discriminating portion 2. The 
10 counted value is outputted to a control portion 5. 

The control portion 5 decides to which group (light or dark) the extracted person's skin color belongs according to 
the counted pixel value and decides, on the basis of the above-mentioned discrimination, a range of color component 
values to be used for further area-extraction. 



A mode representing the decided color range together with a threshold value is outputted to a face-area extracting 



Fig. 8 shows a sequence of operations of the control portion 5 when deciding a color area. At Step S50, a count 
value N is obtained. At Step S51 , a proportion of a face area to an entire image is estimated at 3 to 4% of a total of 
pixels. In this case, the extractable person is considered to have a light skin-color if the counted value exceeds the 
estimated value. At Step S52, a color component area W1 within hl<H<h2 is decided as a specified area. On the 
contrary, the person is considered to have a dark skin color if the counted value does not exceed the threshold. In this 
case : at Step S53, a logical sum of color areas W2 and W3 defined by h2 and h3f rom the control portion 5 is determined 
as a specified area. Generally, h3 is set at 6.0. The area W2 is h3<H^2n and the area W3 is 0<H<h2. As Equation (1 ) 
for deriving a hue value is defined by an inverse cosine, the two areas W2 and W3 have hue values being successive 
to each other at 2k and zero to form a substantially single area. In the face-area extracting portion 4, a face (skin-color) 
imago area according to the threshold value and the mode outputted from the control portion 5 is extracted as, e.g., 
an address for a hue signal read from the frame memory 6. 

Embodiment of the present invention, which corresponds to aforementioned (1) to (4): 

Fig. 9 is a block diagram showing another example of technical means for realizing the present invention. The 
operation of the technical means is as follows: 

This embodiment differs from the aforementioned first embodiment by using a histogram preparing portion 7, a 
noise eliminating portion 8 and a peak detecting portion 9 in place of the primary discriminating portion 2 and the pixel 
counting portion 3 of the first embodiment. 

The histogram preparing portion 7 prepares a histogram of hue values. The histogram may include noise and 
calculation error components. To eliminate these erroneous components, the noise eliminating portion 8 conducts 
smoothing the histogram by using, e.g., a median filter and then cuts off a histogram part whose frequency (the number 
of pixels) is not more than a value T as shown in Fig. 5. The thus obtained histogram is outputted to the peak detecting 
portion 9. 

The peak detecting portion 9 examines whether histogram has a peak in a specified area W1 (this is the same as 
described before with reference to Figs. 5 and 6). The examination result is outputted to a control portion 5. 

According to the examination result, the control portion 5 correlates the histogram with a color area of a person 
with a light skin-color (i.e. , of the white race or the yellow race) if a peak was found in the area W1 , or with a color area 
of a person with dark skin color (i.e., of the black race) if no peak was found therein. Similarly to the case of first 
embodiment, the control portion 5 outputs a decided color-area mode and a threshold to a face-area extracting portion 
4. The face-area extracting portion 4 extracts a face (skin-color) area from a hue signal of an image read from a frame 
memory according to the mode and the threshold received from the control portion 5. 

Embodiment of the present invention, which corresponds to aforementioned (7) to (9): 

The following description relates to an embodiment of the present invention, which treats primarily not with edited 
image (e.g. , a movie picture) but non-edited images taken by a camera. Images of this type allow extraction of respective 
face-areas therefrom by using the same threshold as far as the same person exists in the images. This embodiment 
is intended to use the above-mentioned feature. 

Fig. 10 is a block diagram showing technical means for realizing the present invention. The operation of the tech- 
nical means is as follows; 

A frame memory 6 stores a frame of input original image. 

A color component discriminating portion 10 discriminates a face area in the frame of the original image read from 
the frame memory 6 according to a color component value. The discrimination can be made, for example, by using 
the face-area extracting method described in the first embodiment of the present invention defined in the above (1) to 
(4). The discrimination result is outputted to an input color-space discriminating portion 11. 

The input color-space discriminating portion 11 extracts data corresponding to the same co-ordinates as those of 



15 



portion 4. 



5 



BNSDOCID: <EP 0756426A2J_> 



EP 0 756 426 A2 



10 



15 



20 



25 



30 



35 



40 



50 



55 



m,n e A i°o?r r reshoid is ° u,pu,,ed * a face -- £££££ T area ,n the input coior space is det - 

' ac a-a-aextracting P ortion4^ The «*«»° . portion 5 control the 

to them. I. requires , for examp.e, one second to ^^^thrZ^T^ ^^^ r ^ mlw ^^ Maa 
frames. To achieve real-time face-area extraction T^esla ^^f COrres P onds to a du ^t ion of 30 video 

extracting portion 4 to output 30 successive f'^es ^the Z^ COntr °' T"" 5 therefore ^tructs the face-area 
area from the 31s, and proceeding any processing and extract a face- 

companson with the case of extracting a face-area bv de lnl r I 1 ^ " fedUCe P rocessin 9 time to 1/30 in 
hardware for determining threshold vaLes may be ZomZ^Z * ? * ^ vide °**™- Consequently, a 

The face-area extracting portion TZT^cZoHrl ?, " be I corrtro,led ^ a micro-processor, 
processor extracts a face'a'ea from he original 5 |H ° U,pU,S an orlQ - ^ without 

d IS criminating portion 11 9 9 accord,n 9 to the threshold given from the input color-space 

ofsu™:^ 

video-conferencing apparatus, ft must bZ^at alZ L??" 9 I* * ^ tele P ho " es a " d 

communication, whereby erroneous extraction mToTcur P V a ' ,ema,e W ' th an ° ,her P erson in the middle of 

frames.acurrentthreshold::;:^^ ■ 
portion 5 instructs the frame memory 6 to SfZSZT ST^ (N2> N1 >' ' hiS CaSe ' the ^ 
•s an integer) as shown in Fig. 12 In processfnoTramTs o to ^ T u , *' N ° 2XN2 ,rame No n *N2 C 

method, then a face-area is extracte SSZSSSTi ,1 N2 ' mi h * 7 " « lBto ™ infld b * Iha above^entioned 
N2 is recorded into the frame memorv fi ^HnZ::™^ US ' ng ,he ,hreshold T1 Meanwhile, a frame No. 

to 2XN 2+ N1 are then processed icZnSt^S^^^T T * The frameS N ° N2+N1 +1 

cycle of processing will be repeated tnerefrom a face area by applymg an updated threshold T2. The same 

As is apparent from the foregoing, the present invention offers the foliowing advantages: 
(%tit^ 

(3) It is possible to provide effective means f«f IT* » P " Se rea " Z,n9 the above < 1 > 

detecting peaks thereof w Jh^cSed aria * * — by a "tatog™™ and 

ofcoun,edpixe,shavin 9 aco,oX3^^^ 

image-area extraction specified range. This enables automation of the specified 

(V 6 ,\',! SP ^ 

^component ^c^^SS^^^^^ '™» -cording to the specified 
a threshold value therefrom and perform processfno ooZZZ ™ 1 P into a color-space value, obtain 
threshold value. This can realize a i constable ti^ 1 Z succeed '"9 '^mes by using the obtained 

a specftied area from every f rame S^S^STS 8 ^ ,he C3Se ° f ex,racti "9 

O) It is possible to provide a feature ol h , exponent values obtained for respective frames, 
of the above (7). Th* feature ^"££2^ "^ 7 " ,hreSh °' d Va,Ue in addi,i °" to the invented feature 
an object matter in a series ofTmagos ana re J^SZ V ' ^ "»» Variation ° f 

(9) In addition to the effects of the above (7 )ano ^ n^l^T'*™ 3 SeC < uence of fra ™* 



BNSDOCID: <EP 0756426A2J_> 



6 



10 



15 



EP 0 756 426 A2 

Claims 

1 . A specified image-area extracting method which is capable of deriving a value of a color component having a low 
correlation with luminance from each pixel of an input colored image, determining distributions of all pixels of the 
input image according to every color component value, determining a specified area of each color component 
value on the basis of the determined distributions and extracting a pixel whose color component value belongs to 
the specified area. 

2. A specified area extracting method as defined in claim 1 , characterized in that hue is used as the color component. 

3. A specified area extracting method as defined in claim 1 , characterized in that the distributions of pixels for re- 
spective color component values are determined by preparing a histogram of the color component values, checking 
is made for existence of a peak having a frequency exceeding a preset value in a preset range of values of the 
color component, two specified preset areas are selected to be determined on the basis of the check result. 

4. A specified area extracting method as defined in claim 3, characterized in that the preset specified range of the 
color component values is limited by a fixed upper limit value (h2) and a fixed lower limit value (hi ). 

5. A specified area extracting device comprising arithmetic processing means (1) for calculating a color components 
20 value having a low correlation with luminance for each pixel in an input colored image, a frame memory (6) for 

storing the input colored image and the color component value determined by the arithmetic processing means 
(1), discriminating means (2) for discriminating whether the determined color component value is included within 
a preset specified range of color components values, counting means (3) for counting-up pixels whose color com- 
ponent value being within the specified range according to the discrimination result, specified image-area extracting 
25 means (4) for extracting an image within the selected range of color component values from the frame memory 

(6) and control means (5) for performing control of the above-mentioned means (1 , 6, 2, 3, 4) and calculating 
operations, said control means (5) being provided with facilities for comparing a value counted-up by the counting 
means (3) with a preset value and selecting two specified ranges of the color component values by the specified 
image-area extracting means (4). 

30 

6. A specified area extracting device as defined in claim 5, characterized in that an upper limit value and a lower limit 
value limiting the preset range of color component values are constant values. 

7. A specified area extracting device comprising a frame memory (6) for storing an input colored image, pixel extract- 
35 ing means (4) for deriving a color component value having a bw correlation with luminance for each pixel of the 

input image stored in the frame memory (6), determining distributions of all pixels of the input image according to 
the color component values, determining a specified area of the color component values on the basis of the dis- 
tributions and extracting pixels belonging to the determined specified area, means (7) for preparing a histogram 
of the extracted image pixels in an input color space, means (9) for determining a specified area in the input color 
space from the distributions of the determined histogram, means (4) for applying the determined specified image 
area to an input image of a proceeding frame and extracting pixels belonging to the determined specified area 
from the input image, and control means (5) for control of the above-mentioned means and necessary calculations. 
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8. A specified area extracting device as defined in claim 7, characterized in that the control means (5) periodically 
conducts determination of the specified area in a color space and extraction of pixels belonging to the determined 
specified area by repeatedly applying said specified area to images to be inputted until a new specified area is 
defined. 

9. A specified area extracting device as defined in any one of claims 7 and 8, characterized in that it has facility 
50 adaptable to process image sequence used as the input colored image. 

10. A method of identifying a specified area of an image produced by an image signal having color components and 
a luminance component, comprising processing said image signal to produce a color variable having a reduced 
correlation with said luminance component compared to said color components, analysing said color variable for 

5 $ said image to deduce a type of said specified area, and identifying said specified area in accordance with the 

determined type. 

11. A method of identifying a specified area in a plurality of images produced by an image signal, comprising identifying 
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said specified area in a first of said .mages, analysing said image signal for the identified area to derive a threshold 
therefrom, and using said derived threshold to identify said specified area in another of said plurality of images. 
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FIG.5 
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(54) Specified image-area extracting method and device for producing video information 



(57) A face (skin color) area of a person is extracted 
at a minimized error irrespective of different skin colors 
of the human races. A hue value calculating portion (1) 
determines a hue value from an input image (RGB sig- 
nal). A primary discriminating portion (2) extracts pixels 
whose hue value lies in a specified range defined by lim- 
iting values outputted from a control portion (5). A pixel- 
counting portion (3) counts the extracted pixels. The 



control portion (5) selects a threshold value for extract- 
ing a face area according to the count of the extracted 
pixels and outputs the threshold value to a face-area 
extracting portion (4) which in turn extracts a face area 
according to the threshold value. 

The method and device are usable for video 
processing device, e.g. for producing video information 
suitable to use in video telephones and video confer- 
ences. 
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